\hypertarget{Conclusion \& Future Work}{
}

In this research we investigated how to improve the quality of synthetic image data for material recognition. Recent material databases try to capture the intra-class variation of materials by using different illumination and viewpoint conditions, but the intra-class variation within the material classes is restricted to the database. This seriously limits the research field of material recognition, and manually recording an amount of data to capture the intra-class variation for any material class is a difficult and time-consuming task. However, the synthesis of novel image data could help to bridge the gap of data sparseness.

Previous research showed that using a simple model for photometric stereo to recover surface normals and surface albedo can be used to synthesize enough data to achieve perfect classification accuracy \cite{Targhi}. However, the model was limited to diffuse materials only since the image synthesis was performed with Lambertian reflectance. 

Recovered surface normals and surface albedo from photometric stereo can be used as input for various reflection models and these reflection models can be used to simulate BRDF beyond that of Lambertian reflectance. With this setup, image data for the problem of material recognition can be synthesized using arbitrary illumination and viewpoint conditions to simulate diffuse and glossy/shiny material properties, thus increasing the intra-class variety of material classes.

In our experiments, the different reflection models are tested on image quality in two experiments on two different datasets. The datasets are designed such that one consists mostly of diffuse materials and the other consists of specular materials. 

On the diffuse material dataset, significant improvements were observed using reflection models that obey basic physical laws. Phong reflectance performed similar or worse than Lambertian reflectance. The other specular reflection models implemented, Blinn-Phong and Cook-Torrance performed slightly better than Lambertian reflectance. These models share the concept of microfacets and outperform Phong reflectance, suggesting that more physical based reflectance models are useful in this task of image synthesis. The Oren-Nayar reflection model gives mixed results, performing similar or better than the baseline.

On the glossy/shiny material dataset, no significant improvements were observed in the experiments. The accuracy for the synthetic data is significantly worse for the glossy/shiny material dataset than that of the diffuse material dataset, even though we try various models for specular simulation.

The problem in the setup of these experiments is that of the recovery of surface normals and surface albedo. The quality of these recoveries are directly dependent on the set of image data used in this process and the assumptions made by the photometric stereo method. The algorithm for photometric stereo uses the Lambertian assumption to estimate the surface albedo and surface normals, and for this reason outliers in image data such as speculars are treated as if they are part of the surface albedo. RANSAC or surface normal smoothing are not able to improve the surface normals and surface albedo recovery under the Lambertian assumption. This suggests that more complex methods for photometric stereo are needed that are not constrained to the assumption of Lambertian surfaces to improve the quality of recovery. With the current material databases available for photometric stereo, this is a difficult problem since only intensity information is present. 

For future work, additional analysis of materials could be of use for better estimation of parameters, such as the refraction indices for Fresnel and surface roughness parameters for microfacet models since finding these settings through optimization is computational expensive. The addition of color in material databases could improve the quality of the recovery of surface normals and surface albedo since photometric stereo methods that use color information are more robust to outliers such as speculars in the image data. With improved surface normals and surface albedo, it is likely that the synthetic image data will improve in quality.
